1. Field of the Invention
This invention relates to an image recognition system for recognizing the correspondence between object images contained respectively in two images obtained by two imaging devices.
2. Description of the Relevant Art
Conventionally, a method of detecting a distance to an object is widely known, in which two imaging devices are positioned apart from each other, and a distance to the object is detected based on images of the object contained in two images produced respectively by the two imaging devices. In this method, proper recognition of the correspondence between the object images contained in the respective two images is a prerequisite to accurate detection of the distance to the object.
The SAD (Sum of Absolute Differences) method is a well known method in which one of two images is set to a reference image, and the other of them to a comparison image, and an object image in the comparison image which corresponds to an object image in the reference image is extracted. In this SAD method, the absolute value of a difference between the luminance value of each pixel in a reference pattern which is contained in the reference image and containing the object image, and the luminance value of a corresponding pixel in each pattern which is contained in the comparison image and identical in shape to the reference pattern is calculated on a pixel-by-pixel basis, and the sum of the absolute values is set to a correlation value ERR indicative of the correlation between each two patterns, and then, a pattern minimizing the correlation value ERR (i.e. a pattern having a highest correlation with the reference pattern) is extracted as a corresponding pattern which corresponds to the reference pattern.
However, when the SAD method is employed, if the two imaging devices are different in characteristics (e.g. if the lens of one of the imaging devices is dirty and the luminance values of the corresponding image are generally lowered), the difference between the luminance values of the two object images contained in the respective reference and corresponding patterns is increased even though the two object images are identical in shape to each other, which results in increasing the correlation value ERR and in worst cases, the two object images cannot be determined to be identical.
To solve this problem, a stereo imaging system was proposed in Japanese Laid-Open Patent Publication (Kokai) No. 10-255020, in which a sensitivity-adjusting device is interposed between the two imaging devices and an image recognition system that carries out detection of a distance to an object and other functions, whereby levels of signals output from the respective imaging devices are adjusted for compensation for the difference in characteristics between the two imaging devices.
The sensitivity-adjusting device of the stereo imaging system stores data conversion tables associated respectively with the two imaging devices in a memory, and levels of the signals output from the imaging devices are converted by using the data conversion tables to output the resulting data. Further, to cope with the aging of the sensitivity-adjusting device, values of the data conversion tables are updated by using image data of a specific object at a location within the scope of the common imaging area of two imaging devices.
However, in the above conventional stereo imaging system, it is required to provide the sensitivity-adjusting device between the imaging devices and the image recognition system as well as the memory for storing the data conversion tables. This complicates the construction of the whole system, resulting in a considerable increase in manufacturing costs.
The adverse effects of the difference in characteristics between the two imaging devices can be eliminated by using a normalized cross-correlation parameter as a correlation parameter in place of the correlation value calculated by the SAD method. In the method using the normalized cross-correlation parameter, however, a large amount of arithmetic operation is required, which puts an increased load on an operation system. Therefore, when the image recognition system is installed e.g. on an automotive vehicle for monitoring environmental conditions, it is required to use an expensive arithmetic unit having a high operation speed, which inevitably becomes a main factor of increased manufacturing costs of the automotive vehicle.
It is an object of the invention to provide an image recognition system which is capable of accurately determining identity between object images contained respectively in images obtained by two imaging devices, with hardware having a relatively simple construction and through a small amount of arithmetic operation, even when the two imaging devices are different in characteristics.
To attain the above object, the present invention provides an image recognition system for recognizing correspondence between object images contained respectively in two images obtained by two imaging means, the image recognition system comprising:
corresponding pattern-extracting means for setting one of the two images to a reference image and the other of the two images to a comparison image, calculating a correlation parameter indicative of a correlation between a reference pattern contained in the reference image and containing an object image, and each pattern contained in the comparison image and identical in shape to the reference pattern, by using a luminance value indicative of a luminance of the reference pattern and a luminance value indicative of a luminance of each of the patterns contained in the comparison image, and extracting, based on the calculated correlation parameter, a pattern having a highest correlation with the reference pattern from the comparison image, as a corresponding pattern which corresponds to the reference pattern; and
identity-determining means for determining identity between the object image contained in the reference pattern and an object image contained in the corresponding pattern based on a difference between a contrast value indicative of a contrast of the reference pattern and a contrast value indicative of a contrast of the corresponding pattern.
According to this image recognition system, the correlation parameter indicative of the correlation between the reference pattern contained in the reference image and containing the object image and each pattern contained in the comparison image and identical in shape to the reference pattern is calculated based on the luminance values of the two patterns, and then a pattern having a highest correlation with the reference pattern is extracted from the comparison image based on the calculated correlation parameter, as a corresponding pattern which corresponds to the reference pattern. Then, identity between the object image contained in the reference pattern and an object image contained in the corresponding pattern is determined based on a difference between a contrast value indicative of a contrast of the reference pattern and a contrast value indicative of a contrast of the corresponding pattern. Therefore, it is possible to accurately determine identity between the object images contained in the images obtained by the two imaging devices, respectively, with hardware having a relatively simple construction and through a small amount of arithmetic operation, even when the two imaging devices are different in imaging characteristics.
Preferably, the identity-determining means includes contrast difference component-calculating means for calculating a correlation parameter component corresponding to the difference between the contrast value indicative of the contrast of the reference pattern and the contrast value indicative of the contrast of the corresponding pattern, as a contrast difference component, and determination means for determining identity between the object image contained in the reference pattern and the object image contained in the corresponding pattern, based on a parameter obtained by subtracting the contrast difference component from the correlation parameter concerning the corresponding pattern.
According to this preferred aspect of the invention, the correlation parameter component corresponding to the difference between the contrast value indicative of the contrast of the reference pattern and the contrast value indicative of the contrast of the corresponding pattern is calculated, as a contrast difference component, and identity between the object image contained in the reference pattern and the object image contained in the corresponding pattern is determined based on a parameter obtained by subtracting the contrast difference component from the correlation parameter concerning the corresponding pattern. Therefore, it is possible to more accurately determine the identity between the object images.
Preferably, the identity-determining means calculates the contrast value indicative of the contrast of the reference pattern based on a difference between a luminance value indicative of a luminance of the object image contained in the reference pattern and a luminance value indicative of a luminance of a background portion within the reference pattern, and calculates the contrast value indicative of the contrast of the corresponding pattern based on a difference between a luminance value indicative of a luminance of the object image contained in the corresponding pattern and a luminance value indicative of a luminance of a background portion within the corresponding pattern.
Preferably, the corresponding pattern-extracting means calculates overall contrast values indicative of overall contrasts of the two images, respectively, and sets one of the two images whose overall contrast value is the larger one of the calculated overall contrast values, to the reference image.
According to this preferred aspect of the invention, the overall contrast values of the two images are calculated, and then one of the two images whose contrast value is the larger of the two calculated overall contrast values is set to the reference image. Therefore, it is possible to avoid erroneous identity recognition of the object images due to a difference in contrast between the two images, and hence reliably determine identity of the object images.
Preferably, the corresponding pattern-extracting means calculates each of the overall contrast values indicative of the overall contrasts of the two images by the following steps:
(1) generating a histogram indicative of a frequency distribution of luminance values of each of the two images;
(2) calculating a background peak luminance value at which a portion of the histogram corresponding to a background of the each of the two images exhibits a maximum frequency;
(3) calculating an object peak luminance value at which a portion of the histogram corresponding to the object image contained in each of the two images exhibits a maximum frequency; and
(4) calculating an overall contrast value for the each of the two images based on a difference between the object peak luminance value and the background peak luminance value.
Preferably, the identity-determining means determines that the object image contained in the reference pattern and the object image contained in the corresponding pattern are not identical when the difference between the contrast value indicative of the contrast of the reference pattern and the contrast value indicative of the contrast of the corresponding pattern is larger than a predetermined value.
Preferably, the corresponding pattern-extracting means includes means for setting the reference pattern such that the reference pattern contains a plurality of similar objects positioned close to each other, by clustering.
The above and other objects, features, and advantages of the invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.